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Abstract 

A workshop on the computational fluid dynamics (CFD) prediction of shock boundary- 
layer interactions (SBLIs) was held at the 48th AIAA Aerospace Sciences Meeting. As part 
of the workshop numerous CFD analysts submitted solutions to four experimentally mea- 
sured SBLIs. This paper describes the assessment of the CFD predictions. The assessment 
includes an uncertainty analysis of the experimental data, the definition of an error metric 
and the application of that metric to the CFD solutions. The CFD solutions provided very 
similar levels of error and in general it was difficult to discern clear trends in the data. For 
the Reynolds Averaged Navier-Stokes methods the choice of turbulence model appeared to 
be the largest factor in solution accuracy. Large-eddy simulation methods produced error 
levels similar to RANS methods but provided superior predictions of normal stresses. 


U , V, w 

Nomenclature 

streamwise, transverse and spanwise velocity components 

x, y, z 

cartesian coordinates 

E(f ) 

error metric of / 

Moo 

freestream Mach number 

P 0 

stagnation pressure 

To 

stagnation temperature 

Reg 

Reynolds number based on momentum thickness 

Uoo 

freestream velocity 

< u' 2 > 

streamwise normal stress 

< v' 2 > 

transverse normal stress 

< u'v' > 

shear stress 

5 

boundary-layer thickness 

6 

shock deflection angle 

Ax, Ay, A z 

spacing of PIV measurement grid 

subscripts 

cfd 

computational fluid dynamics obtained quantity 

exp 

experimentally obtained quantity 
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I. Introduction 


A shock boundary-layer interaction (SBLI) is an important and complex fluid dynamic phenomena that 
occurs in supersonic flows. It is one of the primary flow features in supersonic inlets, and if not controlled, 
can have a major impact on inlet performace. Predicting SBLIs and understanding how to control them is 
a critical technology area for development of propulsion systems for supersonic vehicles. The United States 
Air Force, the National Aeronautics and Space Administration (NASA) and numerous aerospace companies 
are developing and researching supersonic vehicles. 

The SBLI considered in this study consists of an oblique shock wave, impinging on a supersonic flow over 
a wall which has formed a turbulent boundary layer. The shock wave imposes an abrupt adverse pressure 
gradient. This pressure gradient negatively affects the boundary layer, thickening the layer and perhaps 
separating the boundary layer from the wall. The interaction increases turbulence levels in the boundary 
layer and typically induces unsteadiness in the shock wave. In wind tunnels and inlets with rectangular 
cross-sections the shock also affects the sidewall boundary layer and corner flows resulting in extremely 
complex three-dimensional flowfields. An excellent description of the phenomena is given by Delery and 
Marvin. 1 In a supersonic inlet the SBLI creates total pressure loss, flow distortion and in the case of mixed- 
and internal-compression inlets the increased potential of inlet unstart (reduced stability). 

The phenomena is complex and difficult to predict using computational fluid dynamics (CFD). Tra- 
ditionally Reynolds Averaged Navier-Stokes (RANS) CFD methods have been used for SBLI predictions. 
RANS methods rely on models to simulate the effect of the turbulence on the mean flow. These models are 
developed and adapted for well-behaved boundary layers and shear layers and can not accurately model the 
complex flow in a SBLI. Large-eddy simulation (LES), a technique which directly computes the large-scale 
turbulence, offers the promise for improved predictions but is just now becoming a practical option for these 
types of flows due to increased computing power. 

A workshop on the CFD prediction of shock boundary-layer interactions was organized by the American 
Institute of Aeronautics and Astronautics (AIAA). 2 4 The workshop was held at the 48th AIAA Aerospace 
Sciences Meeting in Orlando, Florida. The purpose of the workshop was to share prediction methodologies, 
assess the state of the art in SBLI prediction and determine the most promising methods. Four test cases 
with extensive experimental data were identified and the geometry and flow conditions for each were provided 
to the analysts. The experimental data was also provided for two cases - the cases with the weakest shock 
interactions. The data was held back for the two cases with the strongest shock interaction, to assess the 
solutions in a true predictive situation. A key component of this effort was to provide an objective assessment 
of the error for each of the predictions. This paper discusses the assessment of the CFD predictions for the four 
SBLI workshop cases. The assessment includes an evaluation of the experimental uncertainty, a definition 
of an error metric, the results of the error assessment for each test case and a discussion of some trends that 
were identified in the data. 


II. Experimental Data 

The data used in the workshop came from two sources. The general setup of the experiment was similar 
for both (Fig. 1). A shock generator plate was placed in the test section of a supersonic wind tunnel at an 
angle to the freestream flow. This angle, #, is the shock deflection angle, which determines the strength of 
the oblique shock wave. A turbulent boundary layer is present on the floor of the wind tunnel. An oblique 
shock wave is formed by the plate and intersects the boundary layer. Particle Image Velocimetry (PIV) 
measurements of velocities and turbulent quantities were performed in the region of the shock boundary- 
layer interaction. The use of PIV, a relatively new technique, for comparison to CFD represents an advance 
in the assessment of CFD for SBLIs. Typically wall pressure and skin friction data has been used. PIV 
provides a much more detailed look at the flow. Table 1 summarizes the experimental cases. 

The geometries for both test setups were provided in IGES format. The basic layouts of the tunnels 
and shock generator plates, based on the IGES files, are shown in Fig. 2. A comparison of the wind tunnel 
cross-sections including the shock generator plate and boundary-thickness are given in Fig. 3. 
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Figure 1. Schematic of a SBLI in a supersonic wind tunnel 


Organization 

Moo 

9 (deg.) 

To (K) 

Po (kPa) 

Reg 


IUSTI Marseille (UFAST) 

2.25 

8.0 

293 

50.5 

6,900 

open 

University of Michigan (CCAS) 

2.75 

7.75 

293 

101 

6,600 

open 

University of Michigan (CCAS) 

2.75 

10.0 

293 

101 

6,600 

blind 

University of Michigan (CCAS) 

2.75 

12.0 

293 

101 

6,600 

blind 


Table 1. Summary of the experimental cases 



(a) IUSTI (b) UMich 

Figure 2. Wind tunnel and shock generator layout (courtesy of Vishal Bhagwandin, Army Research Labora- 
tory) 
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Figure 3. Comparison of wind tunnel cross-sections 


The two experimental datasets are summarized below. Details of the PIV measurements and their effect 
on the experimental uncertainty are given in the following section (Sec. III). 

A. IUSTI Mach 2.25 — 8.0 degree case 

The first dataset was taken at the Institut Universitaire des Systemes Thermiques Industries (IUSTI) in 
Marseille, France. 5 This data was also used as the test case for the European Union sponsored research 
effort on SBLIs named UFAST. 6 The freestream Mach number, M^, was 2.25 and the deflection angle was 

8.0 degrees. The wind tunnel nozzle has converging/diverging walls on both the upper and lower surfaces 
and straight sidewalls (Fig. 2(a)). The wind tunnel test section was 120 mm high and 170 mm wide. The 
shock generator plate spanned the width of the tunnel, with only a small gap between the plate and tunnel 
sidewalls. The data taken included 2D PIV, stereoscopic PIV, laser doppler velocimetry (LDV) and wall 
pressures. The data used here for comparison is the PIV data on a streamwise plane (a plane parallel to 
the freestream flow and the tunnel sidewall) in the center of the tunnel. The PIV plane was limited to only 
the interaction region. The LDV measurements were used to verify the PIV data. The data provided for 
comparison included: streamwise and transverse velocities, u and u, normal stresses, V< u' 2 > and V < v' 2 > 
and shear stress, — < u'v' >. 

B. UMich. Mach 2.75 — 7.75, 10.0 and 12.0 degree cases 

The remaining datasets were taken at the University of Michigan (UMich.) 7 The UMich effort is part of 
the U.S. Air Force sponsored Collaborative Center for the Aeronautical Sciences (CCAS). The freestream 
Mach number, M^, was 2.75. The shock generator plate was set at three different angles, 7.75, 10.0 and 

12.0 degrees, increasing the strength of the shock and the size of the interaction. The upper wall of the wind 
tunnel served as the converging diverging nozzle (Fig. 2(b)). The wind tunnel test section was 70 mm high 
and 57 mm wide. The shock generator plate was 32 mm wide and spanned the center portion of the tunnel 
(Fig. 3(b)). The narrower plate is an attempt to avoid strong shock boundary- layer interactions on the 
tunnel sidewalls. The test section is narrower than the IUSTI test section and the boundary layer and shock 
generator create a larger percentage of blockage for the tunnel flow. One would expect significant influence 
of the corner flows on the tunnel centerline and a more three-dimensional flowfield. The data consists of 
stereo PIV measurements on a streamwise plane in the center of the tunnel and several spanwise planes (Fig. 
4). The PIV planes include the interaction region and a portion of the freestream flow above. In addition, 
a streamwise data plane was provided to characterize the boundary layer upstream of the interaction. The 
data provided included: streamwise, transverse and spanwise velocities, u , v and re, normal stresses, < u' 2 >, 
< v' 2 > and < w' 2 >, shear stresses, < u'v' >, < u'w' >, and < v'w' > and velocity gradients. Note that 
there is a difference between the two experiments in the form of the turbulent stresses. 
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Figure 4. Streamwise and spanwise planes for PIV measurements for the UMich. data 


III. Experimental Uncertainty Estimation 

This section discusses how the uncertainties in the experimental measurements from IUSTI and UMich. 
were estimated. For the IUSTI experiment, an estimate of the experimental uncertainty was obtained from 
Dussage et al. 8 and for the and for the UMich. experiment, an estimate of uncertainty was obtained from 
Lapsa and Dahm. 7 We also applied an independent method to estimate experimental uncertainty developed 
by Oberkampf and coworkers 9-11 to the UMich. Data. The method is based entirely on statistically analyzing 
the differences in multiple measurements of the various quantities of interest, e.g., u , v, re, as well as normal 
stresses and shear stresses. 

A. IUSTI Data Uncertainty 

Spatial resolution of the stereoscopic PIV technique is limited by the thickness of the laser lightsheet (1 
mm) and the size of the analysis windows. The final effective cell size is 16 (horizontal) x 8 (vertical) pixels, 
leading to the data being averaged over a volume of 1 mm x 0.5 mm, and a light sheet thickness of 1 
mm. No detailed analysis of the uncertainty in the velocity measurements was provided, but the estimated 
uncertainty is quoted as 6 m/s. 5 This value of uncertainty was used in the comparisons of CFD results with 
IUSTI measurements. 

An independent assessment of the accuracy of the PIV measurements was obtained by a number of 
comparisons with LDV measurements for the same conditions in the wind tunnel. It was found that with 
the initial field of view for the PIV that large errors occurred in measured velocities. After the field of view 
was significantly reduced, they found good agreement between PIV and LDV measurements for both mean 
velocities as well as turbulent stresses. 

B. UMich. Data Uncertainty 

The parameters for the PIV set-up at the University of Michigan were similar to those used in the IUSTI 
experiments, except that data were obtained by stereoscopic PIV along a streamwise plane as well as seven 
spanwise planes. The streamwise field of view covers a significantly larger region than each spanwise slice 
and for this reason the PIV window size and operating parameters differ. In the streamwise plane a window 
of 32x32 pixels (the smallest used in the analysis) was approximately equivalent to 1 mm x 1 mm window 
size and the light sheet thickness was 0.5mm. A delay of about 1 fi s was set between the two laser pulses. 
Spanwise planes use a smaller field of view to obtain better spatial resolution and as a result window sizes 
were 0.5 mm x 0.5 mm and the time increment between two laser pulses was around 500 ns. 
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No comprehensive uncertainty analysis was provided for the measurements, nor were the PIV mea- 
surements compared with an independent technique. However, the repeatability of the measurements was 
assessed by comparing the results of different test campaigns performed several weeks apart. The different 
test campaigns involved completely independent forming and positioning of the laser sheets, positioning of 
the stereo cameras, calibration of the camera fields-of-view, and all other aspects associated with set-up and 
calibration of the experiment. Comparison of the two results through the boundary layer showed that the 
mean u component of velocity was indiscernible between the two campaigns, except for the first three points 
near the wall. Comparisons for the mean v component of velocity showed a maximum difference of 9 m/s 
and the average difference through the boundary layer of 3 m/s. 

One particular issue of concern emerged in recent months due to the work of Ethan Eagle and Evan 
Bonny of the Dept, of Aerospace Engineering, University of Michigan, working with the same stero-PIV 
instrumentation. The spanwise high-resolution measurements in particular required very short time intervals 
between the two laser pulses (500 nm) which are at the limit of what is possible with today’s technology. This 
was recognized by the original researchers who measured the actual time delay between two laser pulses at 
the start of the investigation and reported that this was somewhat larger than commanded by the software. 
However, subsequent research by Eagle and Bonny found that the hardware used for the experiments was 
less reliable than previously thought and that the actual delay fluctuates more significantly, giving a velocity 
bias error that is directly proportional to the timing discrepancy. It was found that there is on average an 
additional time delay of up to 50 ns and a shot-to-shot variation of the order of 10-15 ns. The original data 
was obtained with only a single measurement of this timing delay and it is therefore likely that it suffers 
from both a systematic bias error and a random fluctuating error. For the data taken along the spanwise 
planes these errors might be as large as 10-13%, while the streamwise plane data (which was taken with 
larger interval times) may be affected by about half of these estimates. For this reason only the streamwise 
planes are used for comparison in this paper. This topic will be revisited in the next section. 

C. Statistical Estimation Method 

A completely different approach for estimating the experimental uncertainty of measurements obtained in 
wind tunnels has been developed by Oberkampf and coworkers. 9 11 This technique is a statistical approach 
that is based on analyzing comparisons of multiple measurements of quantities of interest. It is usually 
referred to as statistical design of experiments (DOE) because it analyzes the final measured result of the 
quantities of interest, based on specially designed sampling techniques. 12, 13 We were able to apply this 
technique to the published UMich. data and take advantage of the fact that the sampling is derived from 
two different planes. Here we will only give a brief description of the approach, the key computational steps, 
and the results of the analysis. 

The approach compares measurements of each of the quantities of interest that are obtained from both the 
streamwise and spanwise planes. The quantities of interest can be either directly measured quantities, i.e., 
mean velocity components u,v,w, and 'u / ,u / ,u/, or they can be quantities derived from the measurements, 
such as gradients of these quantities or Reynolds stresses. At each of the seven intersections of the spanwise 
planes with the streamwise plane, and for each of the three shock generator angles, the uncertainty in the 
quantities of interest can be estimated. 

This statistical approach could also be applied to comparisons between multiple measurements of the same 
quantity in the same plane. This calculation would independently estimate the repeatability uncertainty that 
was discussed above (Sec. B). It is well known that this type of repeatability uncertainty, also referred to as 
0th order replication, 14 is smaller, sometimes much smaller, than the total uncertainty due to other sources 
in an experiment. Although this calculation was not done for the UMich. experiment, it is planned for future 
work because the repeatability data has been archived. 

When comparisons are made between multiple measurements of the same quantity in different planes, 
then in addition to the repeatability uncertainty just mentioned, one can capture a wider range of systematic 
(bias) uncertainties. Examples of these systematic uncertainties are any changes in the calibration procedure 
of the PIV system (such as spatial positioning of the laser sheet or measurement volume), changes in the 
optical setup of the laser sheet used in the PIV system, and changes in PIV window size or spatial resolution. 
The effect of these types of systematic uncertainties on the quantities of interest can be quantified by using 
streamwise and spanwise measurements because now we have sampled correlated bias uncertainties that 
arise from different populations. That is, anytime in an experiment that the quantities of interest can be 
measured where some source of uncertainty can be altered, then we are sampling from a modified population. 
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The more independent each modified population is, the more accurate we are in estimating the true (total) 
uncertainty. For example, more independence of the sources of correlated bias uncertainties can be obtained 
if one were to measure the velocity components with an LDV technique, or make measurements using a hot 
wire technique in a separate wind tunnel facility. 

To make comparisons of quantities of interest at the intersection of two planes, one must interpolate both 
planes of data to common points in space so that a proper assessment of uncertainty is made. We used 
bi-linear interpolation in each plane of data to obtain each quantity of interest at the following intersection 
points; x is defined by the x value of the spanwise plane of data, y is defined by the y value of the streamwise 
plane of points, and 2 is defined by 2 = 0 (the 2 coordinate the streamwise plane. The spacing of the 
measurement points in the streamwise plane was Ax = Ay = 0.5024 mm and in the spanwise plane it was 
Ay = Az = 0.2879 mm. As a result, the bi-linear interpolation of the measurements required no more than 
half of each of these distances for the interpolation. 

Figure 5 shows the residuals of the u, v, and w mean velocity components for the 21 streamwise and 
spanwise plane intersections (seven spanwise plane intersections for each of the three shock generator angles). 
Since each intersection has 30 - 35 measurement points in the streamwise plane, this yields a total 681 
residuals for each quantity of interest. The residual in Fig. 5 is the difference between the measurement of 
the quantity in each plane, and the abscissa is the average value of the measured quantity in both planes. 
None of the scatter plots show any clear trend with regard to how the residual changes as a function of 
magnitude of velocity, or how it varies with the deflection angle. The v and w plots show a slight trend of 
decreasing residuals as a function of magnitude of velocity. It is believed that the random and bias error 
in the shot-to-shot timing of the two laser pulses mentioned above is responsible for the large scatter in the 
residuals, particularly for the low magnitudes of velocity. 

Also shown in each figure is the mean and standard deviation of the nondimensional residuals. As can be 
seen, the mean of the u residuals is about four times larger than the mean of v and w. The nondimensional 
mean uncertainty for v and w are seen to be very similar. Converting these means to dimensional quantities, 
the iq f, and w uncertainty means are 20.94 m/s, 5.40 m/s, and 3.67 m/s, respectively. As expected, the 
statistical uncertainty in u is larger than the uncertainty in v because (a) u is a factor of 10 larger than v , 
and (b) u is the velocity component that is normal to the spanwise plane. It is well known in stereo-PIV 
measurements that the uncertainty of the velocity component normal to the measurement plane is typically 
two to three times larger than velocity components measured in the plane. The mean uncertainty for v is 
about twice the estimate of the repeatability uncertainty quoted above (Sec. B). Note that even though the 
w velocity component is normal to the streamwise plane, its mean uncertainty is slightly smaller than the 
v component. The v component is the only component that occurs in both the streamwise and spanwise 
planes. 

A similar set of residual plots was generated for each of the measured quantities of interest. Here we 
only show three samples of these results. Figure 6(a) shows the residuals of the streamwise normal stress 

< u' 2 > /U^ versus the normal stress measured at the intersection of each of the planes. Here we observe 
an increase in the uncertainty as a function of the magnitude of the stress. This type of scatter plot for 
uncertainty can be represented as a linear regression function. We require that the linear regression pass 
through (0, 0) so that the only parameter that is computed is the slope of the line. 

Figure 6(b) shows the residuals in the transverse normal stress < v' 2 > /U^ versus the magnitude of 
the average values of < v' 2 > / U in each plane. Here we have a similar linear function to that seen for 

< u' 2 > /U^q. The linear regression indicates that both the uncertainties in < u' 2 > /U^ and < v' 2 > /U^ 
increase with their magnitude. The slope gives an indication of relative magnitude of the uncertainty with 
respect to the value of the quantity and can be converted into a percentage. We see then that this indicates 
the uncertainty in < u' 2 > /U^ relative to its magnitude is just under 20%, while for < v' 2 > /U^ it is 
just under 40%. Figure 6(c) shows < u'v' > /U^ versus the average of < u'v' > /U^ measured in each 
planes. It is not clear that the data support the idea of a linear variation for < u'v' > /E/^, but if a similar 
relationship were assumed it would be of like magnitude. It is important to state that this level of uncertainty 
in turbulence quantities provides a good qualitative measure for turbulence modelers. 

Table 2 summarizes the results of the statistical uncertainty estimation for a number of quantities of 
interest. For quantities of interest that show no clear trend of the scatter plot, the table shows values for 
the mean and standard deviation of the residuals. For quantities that show a linearly increasing trend of 
the uncertainty, the table shows the value of the slope of the linear regression (Y = a + bX). We are still 
analyzing the results to try and explain why certain quantities have a linear trend and some do not. Values of 
uncertainty (the mean of the residuals) that are given in the table are used for the experimental uncertainty 
when comparing CFD results with the UMich. data. 
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(a) Streamwise velocity, u/Uoo 




Figure 5. Residuals of multiple measurements velocity and mean velocity components from the UMich. ex- 
periment 
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Streamwise Normal Stress Residual, A<u’> 2 /U 




(a) Streamwise normal stress, < u ' 2 > /U^ (b) Transverse normal stress, < v ' 2 > /U^q 



Figure 6. Residuals of turbulent stresses versus the average stress measured in each plane from the UMich. 
experiment 
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w/Uqo < M ' 2 > /Ul 


< f ' 2 > /U^o <uW>/Ul 


u/Uoc v/Utt 

mean uncertainty 0.03473 0.008947 0.006081 6.334-10“ 4 2.009-Kr 4 5.317-RT 5 

std. dev. of the uncertainty 0.02745 0.007002 0.004753 0.001087 1.426-Kr 4 6.471-RT 5 

linear regression slope, b 0.1978 0.3804 

Table 2. Summary of the statistical uncertainty analysis 


IV. CFD Solutions 

Eleven individuals from nine different organizations submitted solutions to the workshop for evalua- 
tion. Table 3 summarizes the submissions. The solutions were from both Reynolds Averaged Navier-Stokes 
(RANS) methods and unsteady Navier-Stokes methods, which we will classify here as large-eddy simulation 
(LES). The RANS methods employed a variety of turbulence models and the LES methods used a variety of 
sub-grid models and numerical schemes. Both structured and unstructured grid methods were employed. 


Name 

Organization 

Code 

Method 

Turbulence Model 

Baurle 

NASA Langley 

Vulcan 

RANS 

Menter BSL 

Baurle 

NASA Langley 

Vulcan 

RANS 

Menter SST 

Baurle 

NASA Langley 

Vulcan 

RANS 

k — uj 

Baurle 

NASA Langley 

Vulcan 

RANS 

k — u) + Durbin correction 

Bhagwandin 

Army Research Lab 

CFD++ 

RANS 

Realizable k — e 

Bhagwandin 

Army Research Lab 

CFD+-1- 

RANS 

Spalart-Almaras 

Bhagwandin 

Army Research Lab 

CFD++ 

RANS 

Menter SST 

Duraisamy 

Stanford Univ. 

Joe 

RANS 

Menter SST 

Edwards 

North Carolina State Univ. 

LES 


Edwards 

North Carolina State Univ. 

RANS 

Menter SST 

Georgiadis 

NASA Glenn 

Wind-US 

RANS 

Menter SST 

Jamalamadaka 

Michigan State Univ. 


LES 


Park 15 

NASA Langley 

FUN3D 

RANS 

Spalart-Almaras 

Pirozzoli 16 

Univ. Roma 


LES 


Rumsey 

NASA Langley 

CFL3D 

RANS 

Spalart-Almaras 

Subbareddy 

Univ. of Minnesota 

US3D 

LES 

IDDES 

Tapee 

Nort hr up- Grumman 

RANS 




Table 3. Summary of the submitted solutions 


The majority of RANS submissions modeled the entire wind tunnel from the nozzle plenum to downstream 
of the test section. In these cases the tunnel stagnation temperature and pressure were specified in the 
plenum. The LES calculations typically used a RANS simulation as an initial solution and to provide 
a mean inflow profile to the unsteady simulation of the test section. The LES solutions also required a 
fluctuating component to the inflow profile. This was obtained through boundary layer recycling techniques, 
precursor simulations or theoretical means. 

Each analyst was responsible for post-processing their solutions. This included interpolating their solu- 
tions onto the experimental measurement grids and ensuring that the quantities were provided in the proper 
dimensional form. The analysts were free to choose which cases to run and what data to provide. A total of 
36 solutions were provided and 21 included the turbulent stresses. 
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A. Description of the Error Metric 

In order to objectively evaluate the submitted solutions an error metric, E, was defined. The validation 
metric is a simple average of the magnitude of the difference between the CFD and experiment at every 
point in a measurement plane. The metric is expressed as 

1 N 

E(f) = ^'£\(fefd)n-(fe Xp )n\ ( 1 ) 

n= 1 

For the error in velocities the metric is nondimensionalized by the freestream velocity, Uoq. For the error 
in the turbulence shear stress the metric is nodimensionalized by the square of the freestream velocity, U^. 
The IUSTI normal stresses (\/ < u' 2 > & yj < v' 2 >) are nondimensionalized by [Too and the UMich normal 
stresss (< u' 2 > & < v' 2 >) are nomdimensionalized by For a detailed discussion of the construction 
and use of validation metrics see Ref. 11 

B. Examination of Representative Solutions 

The error metric was computed for all the measured variables and all solutions. In the interest of brevity, 
the comparisons shown here are limited to streamwise and transverse velocities as well as turbulent stresses. 
The velocity derivatives from the UMich experiment were computed using the experimentally obtained 
velocities and differenced on the PIV measurement grid. They do not provide any additional information for 
comparison beyond the velocity data and are a poor comparison to the CFD because the CFD derivatives 
were computed on their own grids, which are much finer. The spanwise velocity, w/Uqq , is also a poor 
quantity for comparison to CFD on the centerline. The tunnel geometry is symmetric about this plane. The 
experimental data reveals that the spanwise velocity was not zero, indicating asymmetry in the flow. CFD 
can not reliably predict such asymmetries and in almost all cases will produce symmetric solutions (in many 
cases symmetry was specified in the computation) with zero spanwise velocity. 

Because the streamwise velocity, u/U oo, is the dominant component most of the attention will be focused 
on it. First, the general characteristics of the flowfields and the error metrics will be presented. For each 
case, the solutions with the lowest (’’best”) and highest (’’worst”) measure of error in streamwise velocity, 
E^/Uoq will be shown. 

C. IUSTI Mach 2.25 — 8.0 degree case 

Figure 7 compares contours of streamwise velocity for the experiment and the best and worst CFD predictions 
of the IUSTI Mach 2.25, 0 = 8.0° case. For this case, the measurement plane extends vertically only to the 
edge of the interaction. The shock wave is not visible. The flow separates from the wall and the boundary 
layer downstream of the separation bubble is significantly thicker. The best solution appears very similar to 
the experiment, although the size of the separation bubble and downstream boundary layer are slightly under- 
predicted. The worst solution shows a much smaller separation and thinner downstream boundary layer. 
Fig. 8 shows the contours of the error metric for the streamwise velocity. The error contours clearly show 
the regions where the predictions deviate from the PIV data. The regions of high error are concentrated 
near the separation bubble. The maximum error in the best solution is approximately 0.2C/oo, while the 
maximum error in the worst solution is near 0.35Uoo. A more traditional comparison is shown in Fig. 9 
where velocity are given at several streamwise locations. Error bars based on the experimentalists’ estimate 5 
are included on the PIV data, but are within the size of the symbols representing the data. Despite the fact 
that the PIV method can not measure very near the wall, it is seems apparent from the profiles that the 
best solution should do a much better job of predicting quantities of interest such as skin friction. 

D. UMich. Mach 2.75 — 7.75, 10.0 and 12.0 degree cases 

Figure 10 compares contours of streamwise velocity for the experiment and the best and worst CFD pre- 
dictions of the UMich Mach 2.75, 0 = 7.75° case. For the Michigan cases, the measurement plane extends 
beyond the interaction to include a portion of the freestream flow. The shock wave is visible, but appears 
smeared in the data. This smearing could be due to some unsteadiness in the shock position and/or the 
volume-averaging nature of the PIV technique. It is not clear if the flow is separated. Both CFD solutions 
under-predict the size of the interaction region and downstream boundary layer thickness. Fig. 11 shows the 
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(a) experiment 



(c) ” worst” solution 



0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 

Figure 7. IUSTI M = 2.25 0 = 8 . 0 °, contours of streamwise velocity, u/Uoo 



(a) ’’best” solution 



(b) ’’worst” solution 

0 0.04 0.08 0.12 0.16 0.2 0.24 0.28 0.32 0.36 

Figure 8. IUSTI M = 2.25 9 = 8 . 0 °, contours of streamwise velocity error, E{u)/Uoo 
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Figure 9. IUSTI M = 2.25 9 = 8.0°, streamwise velocity profiles, u/Uoo 


contours of the error metric for the streamwise velocity. In addition to the error in the interaction region, the 
plots show some error in the location of the shock wave and its reflection. The maximum error in the best 
solution is approximately 0.13£/oo, while the maximum error in the worst solution is near 0.27£/oo. Velocity 
profiles are shown in Fig. 12. Error bars based on the analysis in this paper are included on the PIV data. 
Equivalent figures for the UMich 6 = 10.0° and 6 = 12.0° are shown in Figs. 13 - 15 and 16 - 18. The 
interaction grows larger with increasing deflection angle and the error levels increase for both the best and 
worst predicted solutions. 

E. Prediction of Turbulent Stresses 

The turbulent stresses are computed very differently depending on the numerical approach used. For the 
RANS solutions submitted, the stresses are not explicitly computed and must be derived from the turbulence 
model variables. Typically the Boussinesq approximation is used to compute the components of the stress 
tensor based on the turbulent kinetic energy from a two-equation model. In the case of a one-equation 
turbulence model, an additional approximation for turbulent kinetic energy must be made. An example 
of how this may be done is given by Georgiadis et al. 17 For the LES methods, the stresses are computed 
directly in the simulation. Because the LES solutions have less reliance on modeling approximations, one 
can expect that they will yield better results. 

Two predictions for the turbulent stresses were submitted for the IUSTI case, one RANS and one LES. 
The solutions are compared with the experiment in Figs. 19-21. The experimental data shows very high 
levels of streamwise normal stress in the forward portion of the interaction region. This feature is predicted 
well by the LES and missed by the RANS (Fig. 19). 

The experiment shows that the form and levels of the transverse normal stress differ significantly from 
the streamwise normal stress. The LES solution depicts this change accurately. The RANS prediction of 
the two normal stresses are very similar. This deficiency in the RANS prediction is a direct consequence of 
the approximate method necessary to derive the stresses from the turbulence model. 

The shear stress is shown in Fig. 21. Here, the RANS solution is much improved. The improved prediction 
of the shear stress for RANS is not surprising. RANS turbulence models directly affect the solution through 
the shear stress term in the equations and the models are developed and tuned to accurately model this 
term. 
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x (mm) 

(a) experiment 




0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 

Figure 10. UMich. M = 2.75 0 = 7.75°, contours of streamwise velocity, u/Uoo 



0.05 0.1 0.15 0.2 0.25 0.3 0.35 


Figure 11. UMich. M = 2.75 0 = 7.75°, contours of streamwise velocity error, E(u)/Uoo 


NASA/TM— 201 1-216832 


14 





Figure 12. UMich. M = 2.75 0 = 7.75°, streamwise velocity profiles, u/Uoo 



25 x (mm) 50 


(a) experiment 



(b) ’’best” solution 



(c) ’’worst” solution 


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 


Figure 13. UMich. M = 2.75 0 = 10.0°, contours of streamwise velocity, u/Uoo 
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(a) ’’best” solution (b) ’’worst” solution 


0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 

Figure 14. UMich. M = 2.75 0 = 10.0°, contours of streamwise velocity error, E(u)/Uoo 



Figure 15. UMich. M = 2.75 9 = 10.0°, streamwise velocity profiles, u/Uoo 
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25 x (mm) 50 


(a) experiment 



0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 

Figure 16. UMich. M = 2.75 0 = 12.0°, contours of streamwise velocity, u/Uoo 



0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 


Figure 17. UMich. M = 2.75 0 = 12.0°, contours of streamwise velocity error, E(u)/Uoo 
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Figure 18. UMich. M = 2.75 9 = 12 . 0 °, streamwise velocity profiles, u/Uoo 
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(c) LES solution 
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Figure 19. IUSTI M = 2.25 6 = 8 . 0 °, contours of streamwise normal stress, V < u' 2 >/{Uoo) 2 
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(c) LES solution 
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Figure 20. IUSTI M = 2.25 9 = 8.0°, contours of transverse normal stress, a/ < v' 2 >/(Uoo) 2 



(a) experiment 



(b) RANS solution 



(c) LES solution 


0 0.001 0.002 0.003 0.004 0.005 


Figure 21. IUSTI M = 2.25 6 = 8.0°, contours of shear stress, — < u'v' > /(Loo) 2 
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F. Comparison of all Solutions 

1. IUSTI , Mach 2.25, 0 = 8.0 

The error for all seven predictions (denoted A through G) of the IUSTI case are shown in Fig. 22. For the 
velocity components, the experimental uncertainty quoted by the experimentalists, 1.1 percent of freestream 
velocity, is represented by the dashed line. The average error in streamwise velocity for the CFD predictions 
ranged between 3.8 and 9.0 percent of freestream velocity, which is well above the experimental error. The 
errors in transverse velocity are all near the experimental uncertainty. The relative accuracy of the methods 
is not consistent between the variables plotted here. For example, submission F provides the lowest error in 
streamwise velocity, but provides the highest error in transverse velocity. The error in the prediction of the 
turbulent stresses corresponds to the discussion in section E above. The LES prediction, D, does a superior 
job for the normal stresses and the RANS prediction, G, is better for the shear stress. 



(a) error in streamwise velocity, E(u)/Uoo 


(b) error in transverse velocity, E(v)/Uoo 
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Figure 22. Error assessment of the IUSTI Mach 2.25, 9 = 8.0° case 
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2. UMich. Mach 2.75 - 7.75 , 10.0 and 12.0 degree cases 

The errors for all predictions of the UMich., Mach 2.75 cases are shown in Figs. 23 - 25. The experimental 
uncertainties from the statistical estimation are represented by the dashed lines. For the turbulent normal 
stresses, the mean uncertainty was used, which represents the uncertainty in all planes. A better estimate 
for a given plane could be made using the linear regression and integrating the product of the slope and 
measured value of the stress over that plane. All of the CFD predictions of streamwise velocity and streamwise 
normal stress are near (many are below) the levels of the experimental uncertainty. This suggests that any 
conclusions based on the error data are not valid. But as shown in Figs. 12, 15 and 18, in the interaction 
region the predictions he outside of the experimental error bars suggesting that the use of this error metric 
with the current data can still be useful. 

The computed errors in streamwise velocity for the CFD predictions are given in Table 4. The errors 
grow with increasing deflection angle. This is expected and is a result of the stronger shock wave, larger 
interaction region and more complex flowfield. The lower average errors compared to the IUSTI case is due 
to the inclusion of a portion of the freestream flow in the plane of interest. The freestream portion of the 
flow is much better predicted than the interaction region. Including this portion of the flowfield in the error 
calculation results in a lower average error (Figs. 11, 14 & 17). 

As with the IUSTI case, there does not appear to be a correlation between prediction methodologies and 
the relative error for each variable examined. 

Deflection Angle Min. E(u)/Uoo Max. E(u)/Uoq 


7.75 

0.0240 

0.0404 

10.0 

0.0341 

0.0463 

12.0 

0.0422 

0.0581 


Table 4. Minimum and maximum error in streamwise velocity for the UMich. cases 


G. Discussion of Trends in the Data 

In general the errors in the CFD solutions all fell within a relatively narrow band and no one method 
predicted all variables well. No method stood out as being clearly superior to the others. However the error 
contours and velocity profiles indicate that there is room for improvement in the modeling. RANS methods, 
such as Reynolds stress and algebraic Reynolds stress models may offer improved predictions of the turbulent 
stresses and other flow quantities. Also, the LES predictions suggest that as the method matures results will 
improve. 

The submitted solutions provided a wealth of information. But, because each analyst used a different 
approach there are a large number of variables to consider when attempting to analyze the results. These 
variables include; numerical scheme, grid resolution, grid type, boundary conditions, turbulence model, gas 
model, etc. Based on the information that was submitted with the results and presented at the workshop an 
attempt was made to discern trends in the error data. This is by no means a complete analysis. And it is 
suggested that in order to do a more thorough study, future efforts should include submitted solutions that 
employ common elements such as grids, numerical schemes, etc. 

1. Effect of RANS Turbulence Model 

The clearest discernable effect on the solutions for the RANS submissions was that of the turbulence model. 
There were seven RANS methodologies that were run for all three UMich. flow cases. Fig. 26 shows the 
error in the streamwise velocity prediction for these submissions. For the 7.75 degree case all the Menter 
SST, k — (jo and Spalart-Almaras model solutions are within 0.5 percent of each other. At the larger deflection 
angles, the error in all of the solutions increases and a clear trend is seen. All of the methods show the same 
relative error levels for the 10 and 12 degree cases. The k — u models provide the lowest error and the Menter 
BSL model has the highest error. The Menter SST and Spalart-Almaras models produce similar error levels 
in between the minimum and maximum levels. For the 7.75 degree case several models also exhibit the same 
behavior in the relative error. The error for two Menter SST, the Spalart-Almaras and the Menter BSL 
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(e) error in shear stress, E(< u'v' >)/£^ 

Figure 23. Error assessment of the UMich. Mach 2.75, 6 = 7.75° case 
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(d) error in transverse normal stress, E(< v ' 2 >)/£/J, 



(e) error in shear stress, E(< u'v' 

Figure 24. Error assessment of the UMich. Mach 2.75, 9 = 10.0° case 


NASA/TM— 201 1-216832 


23 


Error in <u'**2> Error in u 



0.06 



(a) error in streamwise velocity, E(u)/Uoo 


(b) error in transverse velocity, E(v)/Uoo 


1.60E-0B 

1.40E-03 

1.20E-0B 

1.00E-03 

8.00E-04 

6.00E-04 

4.00E-04 

2.00E-04 

0.00E+00 


"I 


1.60E-03 


1.40E-03 


1.20E-03 

1.00E-03 

8.00E-04 

6.00E-04 

4.00E-04 

2.00E-04 

0.00E+00 



■ G 

■ H 

■ M 

■ N 

■ O 

■ P 


(c) error in streamwise normal stress, E(< u ' 2 >)/U%q 
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Error assessment of the UMich. 
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12.0° case 


NASA/TM— 201 1-216832 


24 



predictions behave similarly to the 10 and 12 degree cases. The k — u and one of the Menter SST predictions 
do not follow the trend. 

A closer examination of the submitted solutions reveals that the predictions are strongly determined by 
the turbulence model used. This is illustrated by: 1) The three Menter SST solutions were from different 
submissions and used different grids and codes, yet there are only slight differences between them; 2) One 
of the Menter SST solutions, the k — lo solutions and the Menter BSL solution were all obtained using the 
same grid and code, illustrating a strong dependence on turbulence model. 
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(a) UMich. Mach 2.75, 6 = 7.75 (b) UMich. Mach 2.75, 6 = 10.0 
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(c) UMich. Mach 2.75, 0 = 12.0 


Figure 26. RANS turbulence model comparison for the UMich. Mach 2.75 cases. Error in streamwise velocity, 

E(u)/Uoo 


2. Effect of RANS versus LES 

The UMich. Mach 2.75, 6 = 7.75 case was the only case where multiple LES solutions were submitted. The 
LES solutions were the best and worst predictions for this case. Fig. 27 shows the error levels for streamwise 
and transverse velocity for the best and worst RANS and LES predictions. Both methods produce very 
similar error levels. But because RANS methods are far more mature than LES, there is still potential for 
LES to provide superior predictions. In addition, if the prediction of normal stresses is important, LES 
should provide better accuracy. 

3. Effect of Structured versus Unstructured Grids 

Unstructured grid codes are widely used today. Fig. 28 shows the predictions for the IUSTI case grouped by 
structured and unstructured methods. A similar spread in error levels is seen for both methods indicating 
that there is no discernible difference between the grid types. 
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Figure 27 . RANS/LES comparison for the UMich. Mach 2 . 75 , 6 = 7.75° case 
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Figure 28 . Structured/unstructured grid comparison for the IUSTI Mach 2 . 25 , 6 = 8.0° case 
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V. Summary and Conclusions 


A workshop on the computational fluid dynamics (CFD) prediction of shock boundary-layer interactions 
(SBLIs) was held at the 48th AIAA Aerospace Sciences Meeting in Orlando, Florida. The purpose of 
the workshop was to share prediction methodologies, assess the state of the art in SBLI prediction and 
determine the most promising methods. Prior to the meeting four test cases were identified and contributions 
of solutions were solicited from the aerospace community. A total of 36 solutions from 11 contributors 
and 9 organizations were submitted. As part of this effort, an assessment of the submitted solutions was 
performed. This consisted of an assessment of the experimental data, the development of an error metric 
and the assessment of the CFD data using the error metric. 

The experimental data for the four test cases came from two different sources: the Institut Universitaire 
des Systemes Thermiques Industriels (IUSTI) in Marseilles, France and the University of Michigan (UMich.). 
The data used for comparison consisted of velocities and turbulent stresses measured on a plane parallel 
with the freestream flow and the tunnel sidewall. An uncertainty analysis of the experimental data had been 
previously conducted by each of the research groups involved. As part of this work a statistical uncertainty 
analysis of the UMich. data was performed. The analysis compared data from the streamwise and spanwise 
measurement planes at the locations where the planes intersected. Bi-linear interpolation was used to put 
the data at common points in space. The residuals for all the measured quantities were computed and 
the mean values of those residuals is used for the uncertainty estimate. This analysis showed the largest 
uncertainty was for the streamwise velocity and was (in the mean) 3.5 percent of the freestream velocity. 
The uncertainties in the turbulent normal stresses were found to increase linearly with the magnitude of the 
stress. 

An error metric that represents the average error in the predicted quantity for the entire data plane was 
defined. This error metric was used to assess the relative quality of each of the submitted CFD solutions. 
Several variables were examined; streamwise velocity, u , transverse velocity, t?, and turbulent stresses, < 
u' 2 >, < v' 2 > and < u'v' >. It was found that the relative accuracy of the methods was not consistent 
between the variables of interest (ie. a good prediction of u did not imply a good prediction of v or any 
other variable for a given method). Thus, the streamwise velocity was chosen to assess the computational 
methods. For the most part, the submitted solutions were remarkably similar in the error levels. The primary 
differences between solutions were the details of the boundary-layer separation. Because of the many different 
methods used, the lack of consistency for the predictions and the similarity of the solutions, it was difficult to 
identify clear trends or single out any method as superior. In general the errors for all the solutions increased 
with increasing shock strength (increased deflection angle). The choice of turbulence model appears to be 
the dominant factor in the accuracy of the Reynolds Averaged Navier-Stokes (RANS) solutions. Large-eddy 
simulations had similar error levels as the RANS methods, but provided superior predictions of the normal 
stresses. There appeared to be no dependence on computational grid type (structured or unstructured). 

The data shows that there is a need for additional research in the prediction of SBLIs. First, high quality 
experimental data that includes the effects of the sidewall and corner flow interactions is needed and the 
uncertainty of the data should be quantified. RANS turbulence models that provide normals stresses such as 
full Reynolds stress and algebraic Reynolds stress models should be investigated for this problem. Finally, 
LES methods should continue to be advanced as they show promise for this type of flow. 
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